Ignoring Data in Court: An Idealized Decision-Theoretic Analysis

We give a decision-theoretic analysis of a central issue regarding statistical evidence in court: are there circumstances under which it is reasonable to ignore the part of the data that gave rise to suspicion in the first place? We heuristically show that under a minimax/robust Bayesian analysis, this part of the data should in fact be treated differently from any additional data one might have. In some situations, even completely ignoring this part of the data can be a minimax optimal strategy. Lucia de B. is a Dutch nurse who worked in the Juliana children’s hospital in The Hague from 1999 to 2001. She happened to be on duty whenever a patient in her ward suddenly died or suddenly needed to be reanimated. The hospital’s management became suspicious and notified the police. The police then gathered data about Lucia’s shifts in the Red Cross hospital, a hospital where she had worked a few years previously. Thus, these data were only taken into account later, after the investigation against Lucia began. In 2004, the Court of Appeals found her guilty of 7 murders and 3 murder attempts. Statistics played a crucial role in the verdict; a statistician calculated that what happened “could not have been a coincidence”. Despite warnings by the statistician, the court did not need much further evidence to change “not a coincidence” into “murder.” The statistical analysis itself was flawed in several respects. The question is: can we do better? When presenting the case in the UCL evidence seminar (March 20th 2007), I claimed that a purely Bayesian approach is problematic here. From a Bayesian point of view, we would like to determine posterior probabilities that Lucia is innocent or gulty. This requires a prior probability that Lucia is guilty. The problem is that there are a broad range of priors that may be deemed “reasonable,” and these may differ by several orders of magnitude. Therefore, I argued, one should either adopt a “robust Bayesian” approach, adopting a set of priors rather than a single one; or one should adopt a Neyman-Pearson (NP) style hypothesis test, but, to avoid selection bias, one should then ignore the first data set, and only use the second one. The latter proposal generated a lot of resistance. I have now studied both suggestions in more detail, focusing on the question whether it can be sensible to ignore, or at least treat differently, the first data set. Following a suggestion by C. Manski, I have taken a decision-theoretic approach. The result is the present note.